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US  ARMY 


^  RDECOM The  Warfighter  Associate 


•  Software  that  offers  assistance  to  users  (collaboration,  situation 
awareness  development,  and  decision  support)  based  on  a 
model  of  tasks  and  task  performance  in  a  complex,  dynamic 
environment 


•  An  intent  interpretation  engine  -observes  user  actions  (along  with 
the  changing  state  of  the  battle-space)  to  infer  goals  and  needs 

•  Helps  in  planning  (what  should  be  going  on,  given  the  situation) 

•  Knowledge  engineering  is  a  major  part  of  the  architecture  - 
doctrinal  and  subject  matter  expertise,  maintained  separately 
from  the  executable,  can  be  leveraged  to  help  users  dealing  with 
tough  problems 


Leverages  20+  years  /  $30M  of  ongoing  artificial  intelligence  and 
knowledge  engineering  work 


WARFIGHTER  FOCUSED. 


ROECOM 


Command  Post  of  the  Future 
^  _=s^(CPOF) 


•  A  widely  used  command-and-control  platform  used  in  Tactical  Operations 
Centers  to  synchronize/share  work  products  and  achieve  a  common 
operational  picture 

•  Typically  three  displays  per  user  duty  station 

•  Training  and  use  are  challenging  (annual  training  and  field  support  budget 
around  $300  million) 

•  Busy  screens,  displays  not  always  updated  with  current  data,  significant  data  / 
alerts  not  always  there 

•  Cognition  is  distributed 

-  Across  individuals 

-  Across  teams  (multi-team  systems) 

-  Across  environments 

-  Across  artifacts 

One-to-one  human-system  understanding  does  not  scale  directly  to  the 
human-networked  system 
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PILOT'S  ASSOCIATE 


copyright  Applied  Systems 
Intelligence,  Inc.  2008 
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Command  -  Cognitive  Demand 


US  ARMY 


0  RDECOMj^  Intelligent  Aiding  Chaiienge  | 


Cognitive  Performance 


Control  -  Physical  Task  Demand 
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Warfighter  Associate: 
What  does  it  know? 


^All  normal  doctrinal  operations 
^Troops  in  Contact 
^Personnel  Recovery/  downed  aircraft 
^MEDEVAC 

^lED,  VBIED,  SVIED,  P-IED 
^^CASEVAC 
^POO/POI 
^Airspace  ROZ 
v-hvt 

^ISR  asset  and  management 
^Unit  boundaries 
^Joint  and  coalition  coordination 
^Minefields 
/Demonstrations 
^  Air  Threat 


WARFIGHTER  FOCUSED. 


US  ARMY 


^  RDECOM^  WA-How  does  it  know  it? 


•  Brigade  Combat  Team  and  Battalion  Drills  from  Operation  Iraqi  Freedom  /  OEF 

•  Field  Manual  3.0  Operations  Feb  2008 

•  Field  Manual  5.0  The  Operations  Process  Mar  2010 

•  Field  Manual  7-1 5  The  Army  Universal  Task  List  Mar  2005 

•  Joint  Publication  3.0  Joint  Operations  Feb  2008 

•  Field  Manual  6.0  Mission  Command  Aug  2003 

•  Field  Manual  Interim  3.24-2  (FM  90-8,  FM  7-98)  Tactics  in  Counterinsurgency  Mar  2009 

•  4IBCT  3ID  KM_SOP  Final  071630APR10 

•  2009  Army  Capstone  Concept 

•  ARTEP  3-91 -MTP  BCT  Battle  CMD  Staff  200507 

•  FM  3-24.2  TACTICS  INCCUNTERINSURGENCY(FM  90-8,  FM  7-98)APRIL  2009 

•  FM  7-0  TRAINING  UNIT  SAND  DEVELGPING  LEADERS  FCR  FULL  SPECTRUM  CPERATICNS 
FEBRUARY  2011 

•  FM  3-90  TACTICS 

•  FM  1-02  CPERATICNAL  TERMS  AND  GRAPHICS  DEPARTMENT  CF  THE  ARMY  SEPTEMBER 
2004 

•  1st  AIR  CAVALRY  BRIGADE  TACSCP  01  JUL  08 

•  DCD  MIL-STD-2525C  INTERFACE  STANDARD  CCMMCN  WARFIGHTING  SYMBCLCGY  17 
NCVEMBER  2008 

•  GTA  21-08-002  Basic  Combat  Training  Smart  Card  APRIL  2008 

•  Knowledge  from  Roles  from  FCS  program-  22  completed  roles  from  squad  to  Company  level. 

1 1 8  incomplete  roles.  WARFIGHTER  FOCUSED. 
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Plan-Goal  Graph  & 

Intent  Interpretation 


Plan-Goal  Graph  (PGG).  The  goals,  plans  and  actions  that  describe  the 
causality  of  a  domain  are  linked  together  in  a  directed  graph. 

o  Top-level  nodes  of  the  graph  are  abstract  goals,  which  are  linked  to  abstract 
sub  goals.  These  are  then  linked  to  more  concrete  plans  that  might  achieve 
them. 

o  The  bottom-most  input  nodes  are  the  primitive  actions. 


Intent  Interpretation.  Proceeds  by  attempting  to  find  a  path  through  the 
Plan-Goal  Graph  (PGG)  from  an  action  to  a  plan  or  goal  instance  that  has 
previously  been  inferred. 

o  Bottom-up  algorithm  for  intent  interpretation  (observe  user  actions) 

Top-down  planning/concepts  (what  should  be  going  on,  given  the  situation) 

o  Missing  node  instances  needed  to  form  the  path  are  created  hypothetically  as 
the  path  is  discovered. 

o  Intentions  provide  the  context  for  decision-support  to  provide  alerts  and  critical 
information  requirements  to  the  operators. 
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RDECOM 


otential  Benefits  of  Warfighter 
Associate  to  Mission  Command 


Adynamic  real-time  model  of  intentions  provides  mission  command  with  the  ability 
to  engage  in  high  up-tempo  operations  with  support  for  planning  and  plan  execution 
and  fault  tolerance.  It  fosters  a  richer  human-computer  collaboration  and  addresses 
the  complexity  of  mission  command  systems. 

•  Focus  of  attention  for  assessment.  Intent  can  be  used  to  determine  the  critical 
aspects  of  the  environment  to  be  monitored  and  assessed  to  support  dynamic 
planning  and  plan  execution. 

•  Control  of  distributed  planning  and  execution.  Gives  mission  command  greater 
control  over  dynamic  planning  and  plan  execution. 

•  Conflict  detection  and  resolution.  Intentions  can  be  used  to  detect  potential 
conflicts  before  the  appear  in  execution. 

•  Information  management.  Intent  provides  a  basis  for  proactive  dissemination  of 
relevant  information  to  the  tactical  edge  and  to  coalition  partners. 

•  Discrepancy  analysis  and  remediation.  Intentions  are  essential  to  identifying 
discrepancies  in  planning  and  in  execution,  and  in  determining  the  best  means  for 
error  correction. 

•  Agile  Deyelopment  of  Knowledge  .  The  underlying  knowledge  can  be  easily 
modified  to  reflect  new  constructions  of  operational  knowledge.  Allows  for  a 
complete  audit  trail  to  support  the  evolution  of  domain-knowled90i?/7G/yr£/?foc(/5'£D. 
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RDECOM 


Measuring  Performance 
with  the  Warfighter  Associate 


•The  Plan  Goal  Graph  (PGG)  is  a  directed  graph  that  holds  the  goals,  plans, 
and  actions  that  describe  domain  causality 

•The  Concept  Node  Graph  (CNG)  holds  the  model  of  the  battle  space  - 
reported  information  from  sensors,  data  feeds,  command  and  control  system 
messages,  etc. 

•These  graphs  encapsulate  the  knowledge  from  18  Army  publications, 
including  field  manuals,  standard  operating  procedures,  and  written  guidance 
from  operations  in  Iraq  and  Afghanistan. 

•Graphs  also  contain  subject  matter  expertise  on  the  best  responses  to  high- 
intensity  events  (e.g.,  demonstrations,  person  of  interest  sightings,  multiple 
lEDs,  etc.)  that  are  not  recorded  in  formal  publications. 

•As  the  executable  runs,  the  currently  active  nodes  in  the  PGG  and  CNG  are 
noted  and  saved.  The  active  nodes  states  can  be  analyzed  to  show  several 
potential  cognitive  metrics 
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RDECOM^  Campaign  of  Experimentation 


Applied  Systems  Intelligence 


A  campaign  of  experimentation  is  not  just  a  collection  of  experiments  but 
involves  a  series  of  steps  in  moving  from  concepts  to  capabilities  and 
hypotheses  to  understandings.  (Alberts  &  Hayes,  2007) 


•  Goals  The  goal  is  experimentally  address  the  usefulness  of  prototype 
technologies  that  could  be  used  to  assist  battlefield  commanders  and  their 
mission-command  staff  in  the  development  of  situational  awareness, 
decision  making,  and  collaboration. 

•  Reduction  of  data  clutter 

*  Optimization  of  data  relevance 

•  Decision  aiding  and  improved  situation  awareness  (SA) 

*  Synchronization  of  teams  and  collaboration 
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•  Field  Study: 

•  Network  Science:  used  graph  theoretic 
approach  to  examine  mission-command 
collaborative  information  flows.  Metric  was 
the  number  of  self-reported  mission-critical 
collaborations  over  a  two-week  military 
exercise. 


Degree  Distribution; 
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•  Degree  Distributions:  highlights  the  Pareto 
Principle  -  the  top  20%  of  the  mission-command 
staff  is  heavily  involved  in  collaborations. 

•  Our  experimentation  plan  focuses  on  providing 
decision-support  to  three  of  those  individuals: 

S3  (maneuver),  S2  (Intel),  and  FSO  (fires) 

WARFIGHTER  FOCUSED. 
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Laboratory 

Experimentation 


Experimental  Validation  of  Prototype  Technology: 

between-group  experimental  design: 

•  CPOF  alone  (baseline) 

•  CPOF  with  the  Warfighter  Associate 


Metric  Development:  The  Warfighter  Associate  can  also  be  used  to 
develop  useful  metrics  of  performance  rooted  in  an  analysis  of  the 
workflow  (activation  of  goals,  plans,  and  concepts): 

•  cognitive  workload  (objective  measure) 

•  currently  active  plans  &  goals 

•  timing  to  complete  tasks 

•  necessary  collaborations  (shared  plans  &  goals) 
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^  RDECOM 


Data  Analysis  Tools  and 
Metric  Development 


*  Instance  Viewer:  shows  currently  active  nodes  in  the  unified  Plan-Goal-Graph  (PGG) 
and  Concept  Node  Graph  (CNG)  for  each  running  instance  of  the  Warfighter  Associate. 

•  Heat  Map:  view  work  activity  as  a  PGG  'heat  map'  summed  over  epochs. 

•  Cognitive  Workioad:  yields  the  number  of  concurrently  active  PGG  nodes 
over  a  time  frame. 


•  Log  Analyzer:  captures  PGG  instance  start  and  stop  times,  monitor  events,  and 
notification  events  for  each  running  instance  of  the  Warfighter  Associate. 

•  Task  Completion  Timing:  automated  metric. 


•  Shared  Event  Analyzer:  captures  the  time  and  content  of  each  shared  PGG  and  CNG 
node  instance  and  display  a  representation  of  the  shared  events  between  multiple  WA 
Instances. 

•  Force  Synchronization:  metric  reflecting  the  timely  co-activation  of  essential 
tagged  PGG  nodes  by  collaborators  in  response  to  shared  significant  events 
(e.g.  battle  drill)  and  across  time. 

•  Collaborative  Information  Flows:  method  to  tag  hypothesized  collaborative 
information  flows  on  the  PGG  (as  input  and  output  arrows)  and  metric  to  assess 
whether  they  were  supported. 
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0  RDECOM^  Instance  Viewer:  Heat  Map 


|1>|  FSO ,  S2 ,  S3  -  Run  On  Azula  1  -  Data  Analysis  Tool  {version  0.7) 
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l^J  FSO ,  S2 ,  S3  -  Run  On  Azula  1  -  Data  Analysis  Tool  (version  0.7) 


S  OpenConfig  ySave  Config  ;  Manage  ;§  Select  Associates  0  Real  Time  ;0Help 
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0  RDECOM  m  The  Way  Ahead 


•  Make  systems  more  effective  by  addressing  the  “CSI”  challenge  - 
Cognitive  science,  Social  science,  Information  science 

•  Four  bodies  of  work  focused  on  the  problem  space: 

-  Warfighter  Associate 

-  Intelligent  Presentation  Services  -  context-sensitive,  display- 
sensitive,  dynamic  Uls 

-  Metrics  development  -  driven  by  in-house  experimentation 
framework 

-  Socio-cognitive  meta-network  analysis  (team  synchronization. 
Commander’s  Intent  execution,  virtual  interest  groups,  information 
recommender) 
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0  RDECOM Warfighter  Associate 


•  Building  out  additional  roles  (e.g.,  S-6  Network  Management). 

•  Prototyping  interfaces  with  other  systems  of  record  (e.g.,  FBCB2). 

•  Developing  knowledge  bases  and  prototypes  for  lower-echelon 
operations  (company  and  below). 

•  Political,  Military,  Economic,  Social,  Infrastructure,  Information,  Physical 
Environment,  and  Time  (PMESII-PT)  workbench. 

•  Integrating  with  another  set  of  intelligent  agents  that  learn  from  past 
behavior. 

•  Consider  different  visualizations,  triggers  for  behavior,  etc 

^  Experimentation  capabilities  to  enable  assessment  of  cognitive 
performance. 
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0  RDECOM Intelligent  Presentation 


•  Facilitates  the  dynamic  manipulation  of  Uls  in  response  to  changes  in 
context  /  cognitive  challenges  /  user  actions  -  in  a  way  that  is  sensitive 
to  the  available  display  size 

•  Enables  device-agnostic  software  -  one  executable  can  drive  big 
display,  laptop,  handheld  device 

Experimentation  framework  can  be  used  to  quickly  assess  alignment  of 
warfighter  and  systems  capabilities 
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^  RDECOM Metrics  Development 


•  Experimentation  framework  at  Aberdeen  Proving  Ground  facilitates 
collection  of  data  for  analysis 

•  Three-hour  scenario  with  intense  activity  requiring  collaboration, 
situation  awareness,  and  decision  making 

•  Soldiers  (or  civilians)  can  role  play 

•  Advanced  prototypes  intended  for  programs  of  record  or  rapid  response 
prototypes  (mobile  devices,  lower  echelons)  can  be  evaluated 

•  Ongoing  planning  to  insert  prototypes  and  collect  data  at  future  Army 
testing  events 
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Hcognitive  meta-network 
analysis 


•  Using  existing  tools  as  well  as  specific  techniques  to  extract  meaningful 
measures  from  discourse  (chat,  eventually  other  means  of 
communication) 

•  Team  synchronization  and  understanding  of  Commander’s  Intent 

•  Team  strength  /  vulnerability  analysis 

•  Find  people  of  interest  in  the  greater  network 

•  Recommend  information  to  help  solve  problems 

•  Analyzing  the  network  during  periods  of  cognitive  challenge 
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0  RDECOM Transitions  of  the  Technology 


•  The  technology  research  project  overlaps  with  current  technology 
development  project  at  Aberdeen  Proving  Ground 

•  Future  technology  development  projects  focused  on  Mission  Command 
should  use  this  work 

^  Address  difficult  problems  by  using  Associate  Technology  -  develop  and 
share  the  expertise  across  organizations 

Use  Intelligent  Presentation  Services  to  manipulate  smaller  Uls  of 
company  level  and  below  soldiers 

•  Project  Manager  -  Mission  Command  (PM-MC)  is  target  for  Warfighter 
Associate  (integration  into  CPOF) 

•  PM-MC  also  developing  “Command  Web”,  a  web  service  to  support  thin- 
client  functionality  (Intelligent  Presentation  Services  enables  this) 
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Synopsis 


★CERDEC 

us  ARMY- RDECOM 


•  VJhat  is  the  Command  Post  of  the  Future  (CPOF)?  Command-and- 
control  platform  for  synchronizing/sharing  work  products  and  achieving  a 
common  operational  picture. 

o  The  U.S.  Army  will  spend  $224M  for  CPOF  training  and  support  in 
FY201 1 . 


•  What  is  the  Warfighter  Associate?  it  is  an  intent  interpretation  engine 
that  observes  actions  and  the  state  of  the  battle-space  to  infer  user  goals  and 
needs.  ^ 

o  Developed  by: 

o  Knowledge  engineering  is  a  major  part  of  the  architecture 
o  The  emergence  of  genuine  human-centered  computing 


CASEL  experimentation.  Brigade-level  command  and  control  research  of 
key  decision-makers:  Maneuver  (S3)  -  Intel  (S2)  -  Fires  (FSO).  Involves 
mastery  of  technical,  scenario,  and  analytical  elements. 
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Intent  Interpreter: 
Knowledge  representation 


Actions.  Entities  interact  with  the  environment  by  performing  actions. 
Actions  represent  the  primitive  manipulations  of  the  environment  and 
are  the  directly  observable  inputs  to  the  intent  interpreter. 


Scripts.  Scripts  represent  procedural  knowledge  as  weakly  ordered 
sequences  of  events.  Events  are  composed  of  actions  and  constraints. 
Scripts  are  segmented  and  provide  serial  and  parallel  execution  paths. 


Pians.  Plans  represent  abstract  methods  that  might  be  expected  to 
achieve  on  or  more  goals.  While  a  plan  may  have  a  script  associated 
with  it  that  defines  its  nominal  execution,  this  is  not  required. 


Goais.  Goals  represent  desired  states  of  the  environment  that  may  be 
achieved  at  some  time  in  the  future  as  a  result  of  plan  execution. 
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•  The  intent  interpretation  model  combines  a  procedure  follower  with  a  search  process 
to  provide  efficient  interpretation  of  both  procedural  and  non-procedural  behaviors. 

•  Each  entity  is  assumed  to  have  multiple  simultaneous  intentions  that  are  dynamically 
changing  as  the  entity  interacts  with  its  task  environment. 

•  As  each  observation  is  processed,  its  likely  interpretation  is  found,  but  the 
interpretation  is  repeatedly  refined  as  new  data  is  received. 

Geddes  &  Lizza  (2001) 


